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The Academic Pathways of People Learning Engineering Survey (APPLES2) was administered in 
Spring 2008 to undergraduate engineering students at 21 American universities. Students took the 
10-minute online survey that asked mainly multiple choice questions related to their undergraduate 
engineering experience. A final optional open-ended question asked, Is there anything else you want 
to tell us that we didn’ t already cover? This paper explores the responses from survey participants to 
this open-ended question. 

Implications of Findings 

It is illuminating to read student responses and hear their 
voiced passions, concerns and experiences that could not be 
easily captured solely in a standard multiple choice question 
format. The situations and barriers that students describe in 
their comments are wrenching at times. But for each 
extremely bad or frustrating comment there is usually a paired positive one. In reviewing the student 
responses it seems that for each “[my institution] sucks” there is a complimentary “[my institution) 
rocks.” Due to the very nature of the question asked and examined in this paper, perhaps the wide 
range of responses is to be expected. 

These open-ended responses provided a rich addition to the emerging quantitative research findings 
from the APPLES2 survey instrument. Issues important to students such as advising and gender were 
not subjects that were probed as part of the multiple choice survey questions. Additionally, there are 
a considerable number of untapped student experiences outside the classroom to be understood. 

These open-ended student responses add qualitative descriptions and substantiations to the other 
survey data that was collected, with personal and sometimes passionate descriptions of the students’ 
experience, and will inform further iterations of the survey instrument. 

Methods and Background 

This paper analyzes student responses to the open ended question “Is there anything else you want to 
tell us that we didn’t already cover?” 4,266 participants from 21 sites submitted survey responses to 
the APPLES2 deployment and 37 percent, or 1578 of the participants provided free form responses. 
The remaining 2688 participants, or 63%, gave no response and left the question response box blank. 


It is instructive to read student 
responses and hear their voiced 
passions, concerns and experiences 
that could not be easily captured in 
standard survey format. 


Survey responses were removed from the data set if they had no applicable content. To maintain 
student anonymity within individual school reports, a data cleaning schema was developed and 
implemented for each set of school survey records. Student names were redacted and replaced with a 
generic placeholder indicating the removed information, i.e., [name], with brackets and italicized 
words as to indicate the edit. The same was done for any possibly identifying organizations, 
companies, or other affiliations. For further cross-school analysis of student responses (and to 
prepare the same data set for archiving), the aggregate data set was anonymized for school-specific 
information such as school name, individual names, course number and names; other identifiers were 
also given generic replacements such as [institution], [name], [introductory computer science 
course], etc. 

Student responses were graded by two coders for negative and positive values of the comment. A 
scale of 1 through 5 was used to indicate how negative (criticizing), neutral, or positive 
(complementing) the response was. A value of 1 was used to code a very negative response, 2 
slightly negative, 3 neutral, 4 slightly positive, and 5 very positive. Comments that were judged in 
the very categories (1 or 5) used exclamation marks, rather damning or laudatory language and 
otherwise conveyed much displeasure or excitement about the topic described. Comments without 
tone or opinion were marked as neutral. Comments that were only slightly negative or positive were 
categorized as 2 or 4. To make meaning of students responses, comments were read multiple times to 
generate and refine an emerging thematic coding scheme. These topics were grouped by whether 
they were comments about School or Individual Beliefs. Issues emerging from comments at the 
School level are those that could be addressed by institutions but not easily by the student. Similarly, 
issues emerging from comments at the Individual Beliefs level are those that may be more difficult to 
alter if the institution attempted to address them. 

What We Found 

Student responses (scored along the 1 to 5, negative to positive, scale) were coded to a schema that 
included the following 12 codes: advising, co-op, gender, social, teaching (curriculum), and teaching 
(language) as part of the School theme; calling, challenge, future, lifestyle, money, and 
understanding as part of the Individual Beliefs theme (for more detail, please see the full paper at the 
link below). 

The topics under the School theme were mostly found at the extremes of the scale of 
positive/negative comments. For the topics under the Individual Beliefs theme, responses were 
mostly found to be neutral on the scale of positive/negative comments. The number of items per topic 
is generally similar with an exception of the Teaching (curriculum) topic which had 324 comments. 

The School themed topics are generally more negative than the Individual Beliefs topics. 
Interestingly, both Co-op and Money are exceptions in this data set. It may be that these two topics 
are much more concrete than the other more abstract items or that, in reflection, the categorization of 
each should be reconsidered. In other words, finding benefit from experiencing a co-op and being 
worried about the financial overload of tuition could be construed as a miscategorization. 

These open-ended responses provided a rich addition to the emerging quantitative research findings 
from the APPLES2 survey instrument. 
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